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''/The  model  on  which  this  research  is  based  proposes  that  an 
instructional  strategy  has  a direct  effect  (on  the  objective  it 
is  directed  toward)  and  indirect  effects  (or.  related  objectives) . 


Most  training  situations  aim  at  teaching  a group  of  related 
objectives.  Follv-ving  the  recarmendaticn  of  current  learning 
theory,  a maximally  effective  training  sequence  would  consist 
of  a separate  instructional  strategy  to  teach  each  objective. 
.This  is  costly  in  time.  The  model  suggests  that  it  might  not 
be  necessary  to  use  a separate  strategy  to  train  each  objective 
if  indirect  effects  allow  us  to  achieve  more  than  one  abjective 
at  a time.  Acconplishing  more  than  one  objective  at  a time 
results  in  more  efficient  training. 

Seven  objectives  associated  with  mastering  vocabulary 
concepts  were  identified  and  an  instructional  strategy  designed 
for  each.  Two  of  the  strategies  were  tried-out.  Early  results 
shew  that  seme  methods  produce  high  levels  of  achievement  on 
more  than  one  objective.  The  goal  of  our  continuing  research 
is  to  select  the  most  efficient  combination  of  strategies  that 


will  produce  the  desired  high  levels  of  achievanent. 
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SUMMARY 


This  program  of  research  has  set  out  to  address  the  issue  of 
efficiency  in  .instruction  in  teaching  vocabulary  concepts.  There 
are  a number'  of  performance  outcomes  associated  with  "knowing"  a 
word;  we  have  identified  seven  such  outcomes  and  have  developed 
seven  instructional  modules/  one  targeted  toward  each  cut  cane. 
Instruction  of  this  sort  should  be  effective,  but  not  necessarily 
efficient,  in  the  sense  that  a great  deal  of  learning  time  would 
be  spent  going  through  each  of  the  seven  modules  in  order  to  learn 
each  outcome.  We  hypothesize  that  it  is  not  necessary  for  sub- 
jects to  receive  instruction  via  all  seven  modules  in  order  to 
achieve  acceptable  levels  of  performance  on  the  criterion  tests 
measuring  each  outcome.  We  hypothesize  (in  a quantitative  repre- 
sentation of  the  design  problem)  that  each  instructional  module 
will  have  a direct  effect,  shown  by  performance  on  its  related 
criterion  test,  and  that  it  will  also  have  an  indirect  effect, 
smaller  than  its  direct  effect,  cn  the  criterion  tests  measuring 
the  other  learning  outcomes.  If  this  is  true,  seme  smaller  combina- 
tion of  teaching  modules  than  the  full  set  should  produce  acceptable 
levels  of  performance  cn  the  outcome  measures,  at  a lower  cost. 

The  purpose  of  our  program  of  research  is  to  evaluate  the  hypothesis 
and  to  determine  the  optimal  combinations  of  modules  to  teach 
vocabulary  concepts  should  the  hypothesis  be  confirmed. 

The  seven  instructional  modules  are  described  in  the  report, 
in  terms  of  their  instructional  routines;  i.e. , the  sets  of  stimuli, 


responses  and  feedback  each  contains.  The  rationale  for  each 
module  is  presented,  citing  whether  the  module  evolved  from  an 
instructional  design  point  of  view  or  whether  it  is  an  applica- 
tion of  psychological  theory. 

To  achieve  relatively  tight  control  over  student  learning 
processes,  a CAI  (Computer- Assisted  Instruction)  mode  is  desirable 
to  instrument  the  research.  We  have  accepted  delivery  of  two 
VT-52  DECSCQPE  terminals  and  two  sets  of  Vadic  Corp.  modems. 

The  first  stage  of  research  requires  calibrating  the  size  of 
the  direct  and  indirect  instructional  effects;  later  stages  will 
require  studying  how  the  combined  use  of  the  methods  confirms  or 
discanfirms  the  decision  theory  equations  for  optimal  combinations 
of  modules.  In  summary,  the  main  accomplishments  to  date  of  this 
research  endeavor  include:  1)  A formalization  of  the  decision 
theory  framework  that  guides  this  research;  2)  Descriptions  of 
the  seven  instructional  modules  and  a rationale  for  each;  3)  The 
introduction  of  a cognitive  psychology  perspective  on  the  design 
research  so  as  to  aid  understanding  of  learning;  4)  The  detailed 
design  of  the  seven  instructional  modules  plus  the  pretest; 

5)  Development  of  the  instructional  database  for  24  words  for 
six  of  the  modules;  6)  Design  of  the  criterion  tests;  and  7) 
Preliminary  testing  of  the  effectiveness  of  two  of  the  instruc- 
tional modules. 

A tryout  of  the  DEFINE  and  CLASSIFY  modules  (in  adapted 
pencil  and  paper  form)  was  conducted  to  determine  the  existence 
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of  indirect  effects.  All  of  the  instructional  routines  making 
up  each  module  were  used.  Twelve  students  were  assigned  randomly 
to  each  of  two  instructional  groups.  Comparison  of  data  obtained 
on  six  of  the  post- tests  shewed  evidence  of  indirect  effects. 

When  the  quality  of  perfomanoe  on  three  post-tests  was  octnpared 
with  the  quality  of  performance  on  the  post-test  that  matched  in- 
struction it  appeared  that  Define  instruction  led  to  greater  in- 
direct effects  than  Classify  instruction.  The  differences  were 
not  statistically  significant  but  the  trends  suggest  different 
sized  indirect  effects  for  different  modules  arid  this  will  be  one 
of  the  questions  relating  to  optimal  carbinations  of  modules  which 
will  be  pursued  in  forthcoming  studies.  We  have  also  decided  to 
assess  indirect  affects  under  conditions  of  instructional  treat- 
ments with  pre-defined  proficiency  levels  rather  than  under  con- 
ditions defined  in  terms  of  a fixed  amount  of  time  or  number  of 
items. 
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The  CAI  development  that  we  wish  to  report  in  this  paper  is 
directed  toward  enhancing  vocabulary  knowledge  of  lew-verbal 
skills  college  students.  The  instructional  design  of  the  CAI 
system  is  a sinple  pretest,  instruction,  posttest  design.  The 
words  to  be  taught  have  been  selected  from  existing  frequency 
lists  with  special  attention  to  their  "power"  (occurrence  in  a 
wide  range  of  reading  mterials) . The  sites  at  which  the  in- 
struction will  be  used  will  be  developmental  reading  laboratories 
at  the  campuses  of  a university  and/or  a cemnunity  college. 

In  this  paper,  we  wish  to  report  the  progress  we  have  made 
on  the  design  of  cur  CAI  system  and  the  results  of  two  studies 
that  have  helped  us  sharpen  the  methodology  of  our  instructional 
research  in  vocabulary  and  that  have  provided  preliminary  infor- 
mation on  the  relative  effectiveness  of  two  of  the  instructional 
•.nodules. 

Theoretical  Framework  and  Research  Objectives 

We  are  developing  a variety  of  strategies  for  instructing 
the  words  that,  according  to  the  pretest,  a given  student  may 
need  to  learn.  Seme  of  the  strategies  we  have  built  are  similar 
to  those  found  within  developmental  reading  materials;  others 
are  crnite  different  (see  later  section  for  detailed  descrip- 
tion of  the  strategies) . We  have  specified  a number  or  per- 
formance objectives  as  the  goals  of  our  vocabulary  instruction 
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and  each  of  the  strategies  is  focused  cn  one  of  these  performance 
objectives.  Cur  objectives  (Ck  ) refer  to  terminal  behaviors 
such  as  being  able  to  define  a word,  use  it  in  a sentence, 
classify  instances  of  the  word  properly  and  so  forth. 

There  are  seme  particularly  intriguing  aspects  of  our 
instructional  situation.  Previous  research  (e.g. , Johnson  and 
Stratton,  1966)  has  shown  that  one  of  the  likely  characteristics 
of  several  of  our  instructional  strategies  is  that  they  may  pro- 
vide more  instructional  returns  than  cne  might  ordinarily  expect. 

It  is  likely  that  a given  strategy  will  have  an  effect  on  the 
oh  ective  it  is  directly  related  to  and  on  other  objectives. 

Each  of  our  instructional  strategies,  therefore,  appears  to  con- 
tribute to  the  acquisition  of  more  than  one  objective.  Thus, 
we  might  restate  the  matter  by  saying  a strategy  has  both  a 
direct  effect  (on  the  objective  it  is  directly  related  to)  and 
an  indirect  effect  (cn  other  objectives) . The  interesting  problem 
as  we  see  it  is  to  develop  rules  for  deciding  the  best  mix  of 
strategies,  when  certain  levels  of  achievement  on  tests  of  the 
objectives  are  desired.  The  rules,  of  course,  would  be  formulated 
to  take  into  account  both  the  direct  and  indirect  effects  of  the 
given  strategies.  The  rules  will  also  take  into  account  the 
instructional  "costs"  associated  with  given  individual  modules 
or  combinations  of  modules.  We  currently  view  "costs"  to  be  amount 
of  student  time. 
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In  collaboration  with  Guy  Groen,  we  have  developed  a quanti 
tative  representation  of  the  design  problem  we  face  and  it  takes 
the  following  form: 

1.  There  are  n learning  outcomes,  0^  (objectives). 

2.  For  each  0^,  there  exists  a method  and  a criterion 
test,  CT 

3.  yields  a higher  score  on  CTV  than  does  on  CTj. 
Garment:  Thin  assurption  reflects  the  admonitions  of 
Gagnd  and  others  that  instruction  that  is  highly  con- 
sistent with  tests  that  measure  its  effects  leads  to 


greater  achievement  than  instruction  that  is  less 
consistent.  We  shall  drop  the  assumption  if  it  does 
not  hold  in  our  particular  application.  This  v.ill  in 
turn  mean  that  we  can  allow  arbitrary  selection  of 
M^'s  when  composing  prescriptions. 

4.  Each  has  an  associated  cost,  c^. 

5.  Each  Clh  has  a lower  bound  of  acceptable  performance, 


6.  Each  has  a direct  effect  > o on  Clh  and  an 
indirect  effect  > o on  each  CTj  (j  ^ i)  such  that 


{S} 


CTi 


jeS 

when  S is  the  set  of  methods  being  used. 
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Ccnment  it  The  a^'s  are  supposed  to  be  fixed  effects. 

Thus , CT . varies  depending  on  which  a . . ' s 
are  in  the  linear  combination. 

Cement  ii:  Equation  (6)  is  to  be  considered 
locally  a aeptable. 

Clearly,  as  the  formulation  shows,  the  central  problem  for 
this  stage  of  our  research  is  how  to  determine  the  's,  since 
they  are  the  determiners  of  level  of  achievement  on  a given 
criterion  test.  We  are  adopting  two  approaches  to  determining 
the  a^j's.  One  approach  we  call  bottom-up,  which  means  that  we 
shall  determine  the  's  from  the  pattern  of  criterion  test  scores 
that  are  obtained  after  students  receive  instruction  via  a given 
set  of  modules.  The  second  approach  we  call  top-down  and  it 
assumes  that  each  of  the  modules  has  a structure  that  can  be  de- 
noted in  various  ways.  Cne  model  of  structure  that  vie  have  worked 
out  so  far  involves  simply  decomposing  the  modules  into  their 
instructional  routines.  Each  module  can  be  analyzed  into  com- 
ponent routines  serving  different  psychological  functions,  and 
some  routines  are  common  to  sane  modules. 

Both  approaches,  bottem-up  and  top-down,  lead  to  intr  esting 
testable  predictions  about  patterns  and  levels  of  achievement 
on  the  criterion  tests  as  a function  of  what  modules  are  used 
in  the  students  instructional  prescription.  We  shall  not  explore 
the  full  set  of  predictions  that  can  be  generated  from  the  model 


here;  we  shall,  however,  show  one  interesting  consequence  of  (6) 

on  the  previous  page*  It  is  possible  to  derive  the  theorem: 

7 {1,2}  {1,2}  {1}  {2} 

cr1  + cr2  > cr1  + cr2 

The  theorem  can  be  shown  to  be  true  in  this  manner: 


Thus, 


{1}  {2} 
CT1  = a11  and  CT2  = 


a 


22* 


{1}  {2} 

CT1  + cr2  = all  + a22‘ 


But, 

{1,2}  {1,2} 

CT1  + CT2  = °11  + °22  + °12  + a21? 
and  so, 

{1,2}  {1,2}  {1}  {2} 

CT1  + CT2  > CTi  + CT2* 


The  above  theorem  is  tested  in  an  experiment  comparing  student 
achievement  sunned  on  two  criterion  tests  as  a function  of  in- 
structional modules  (1  or  2 only  vs.  1 and  2)  received.  Our 
research  in  the  forthcoming  months  will  be  directed  toward  test- 
ing tneorems  such  as  (7) . We  shall  initially  be  concerned  with 
developing  our  prescriptive  ules  as  they  apply  to  groups  of 
students;  eventually  we  hope  to  be  able  to  apply  the  formulation 
we  have  developed  to  the  learning  of  individuals,  for  one  might 
reasonably  expect  individuals  to  differ  in  their  propensity  to 
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transfer  (i.e. , the  magnitude  of  indirect  effects  ought  to  vary 
among  individuals) . 

Exploration  of  the  previously  described  model  t*  t we  think 
governs  construction  of  instructional  prescriptions  in  vocabulary 
shews  that  it  has  seme  interesting  consequences.  The  key  concept 
of  the  model  is  the  existence  of  indirect  effects.  If  indirect 
effects  do  not  exist,  then  the  question  of  what  constitutes  the 
most  appropriate  prescription  is  uninteresting,  because  in  order 
to  develop  each  one  of  a set  of  performance  objectives,  each  of 
the  methods  must  be  used.  Therefore,  the  major  purpose  of  the 
first  otages  of  our  research  has  been  to  demonstrate  that  in- 
direct effects  exist.  Of  course,  to  enhance  the  chances  that  the 
methods  we  develop  do  in  fact  produce  indirect  effects,  we  have 
incorporated  the  methods  used  in  the  1966  study  which  were,  in 
turn,  based  upon  methods  found  in  the  developmental  reading 
literature. 

Our  purpose  in  this  paper  is  to  describe  the  seven  in- 
structional modules  we  have  developed  in  terms  of  the  design 
guidelines  used  to  create  them  and  in  terms  of  their  oorrponent 
routines.  We  also  include  displays  of  the  instruction  as  it  will 
appear  on  the  CRT,  for  visual  displays  show  instructional 
task  content  in  ways  that  are  not  easily  captured  in  text  des- 
criptions. We  shall  also  describe  seme  research  of  a preliminary 
nature  that  bears  on  the  question  of  the  existence  of  indirect 
effects. 
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This  section  will  describe  the  DEFINE,  CLASSIFY,  WORD 
RELATIONS , WORDS  IN  CONTEXT,  WORD  LINE,  DISCOVER  and  EQUIVALENTS 
modules.  Each  module  carprises  smaller  segments  of  instruction 
called  routines  - each  routine  corresponding  to  a change  in  the 
stimulus-response  situation.  The  routines  were  designed  to  follow 
a shaping  procedure,  i.e. , gradually  reducing  the  stimulus  support 
for  the  terminal  response  as  the  module  precedes.  Routines  are 
of  two  kinds:  expository  and  inquisitory.  Sane  routines  occur 
in  more  than  one  module.  Table  1 lists  and  briefly  describes 
all  routines  for  all  seven  modules  and  can  be  consulted  to  com- 
pare them.  It  will  be  noted  that  seme  routines  use  a response 
contingent  criterion  for  exit;  others  do  not. 

Insert  Table  1 about  here 

Define  Module 

The  define  module  consists  of  seven  routines. 

Routine  1;  Pronounce.  This  routine  presents  the  target 
word  and  a phonetic  pronunciation  key,  with  the  direction  to  the 
student  to  pronounce  the  word.  This  routine  begins  six  of  our 
seven  modules. 

Routine  2:  Definition.  An  extended  definition  of  the 
target  word  is  shown,  emphasizing  the  critical  attributes  by 
separating  them  visually  from  the  rest  of  the  display.  The 
student  is  instructed  to  read  the  definition. 
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Routine  3;  Type  from  Memory.  While  the  definition  remains 
on  the  screen  the  student  is  instructed  to  look  carefully  at  the 
word  until  ready  to  type  it  from  memory.  The  student  then  types 
the  word  after  the  stimulus  disappears  from  the  screen.  Feed- 
back either  confirms  a correct  response  or  - in  the  case  of 
an  incorrect  response  - presents  the  word  again,  which  is  re- 
typed by  the  student  with  the  word  in  view.  (Figure  1 presents 
screen  displays  for  DEFINE,  Routines  3,  5 and  6) . 

Insert  Figure  1 about  here 

Routine  4;  Definition  with  Attribute  Labeling.  First  the 
form  class  of  the  target  word  is  identified  along  with  a general 
statement  of  what  a definition  of  this  form  class  should  con- 
sist of.  Then  a short  form  of  the  target  word's  definition  is 
presented  with  the  parts  labeled  according  to  the  preceding 
statement. 

Routine  5;  Multiple  Choice  Paraphrase.  In  this  routine 
part  of  a definition  paraphrased  from  the  original  definition 
appears  an  the  screen;  it  is  incomplete  because  one  of  the 
critical  attributes  is  left  blank.  The  student  selects  from 
three  choices  the  paraphrase  of  the  missing  critical  attribute. 
Feedback  for  an  incorrect  response  tells  the  student  to  pick 
again.  This  routine  is  repeated  for  the  number  of  critical 
attributes  contained  in  the  target  word. 

In  this  routine  an  option  is  introduced.  The  definition 
from  Routine  4 can  be  either  present  or  absent  while  the  multiple 
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define 


CLASSIFY 


Routine  3:  Type  word  from  memory 


r 


Sometimes,  you  mey  meet  e person  who  attempts 
to  win  fevor  or  edvencement  by  flattering  persons  of 
influence.  This  kind  of  person  who  wants  to  get  ahead 
by  soft-soaping  is  celled  e SYCOPHANT. 

Thus,  e SYCOPHANT  is:  a person 

who  tries  to  get  ahead 
by  buttering  up 
influential  people. 

Study  the  new  word. 

Press  return  when  you  ere  reedy 
to  spell  it  from  memory  > 


Routine  S:  Multiply  choice  paraphrase  of  each  of  the 
definition's  critical  attributes 


r 

Pick  the  phrase  thet  correctly  restates 
part  of  the  definition. 

\ 

A sycophant  is:  anyone 

who  tries  to  (**??**) 
by  apple-polishing  people  in  control. 

1 ) improve  his  or  her  2)  help  other 
position  in  life  people 

Type  its  number .> 

3)  hurt  his  or 
her  friends 

Routine  6:  Multiple  choice  paraphrase  of  ell 
critical  attributes 


A sycophant  Is  (GEN  CLASS)  who  tries  to 
(SPEC  QUAL  1)  by  (SPEC  QUA L 2). 

SOMEONE  HARM  HIS  FRIENDS 

ANYTHING  BE  HELPFUL 

ANYONE  OR  ANYTHING  BECOME  SUCCESSFUL 

PRAISING  POWERFUL  PEOPLE 
WORKING  HARO 
BORROWING 

Complete  the  definition  by  typing  the 
correct  kay  terms.  Press  return  after 
each  key  term. 


t sycophant  is  > 


Routine  3:  Matched  Example  and  Non-example 


r ^ 

This  is  an  EXAMPLE  of  a sycophant: 

a sterlet  buttering  up  a well-known  film  director 
in  order  to  get  e leading  role 

This  is  NOT  AN  EXAMPLE  of  a sycophant: 
a sterlet  dating  e film  director  because 
sha  likas  him 

In  which  instance  it  the  sterlet  trying  to  get  e favor? 

Type  EXAMPLE  or  NONEXAMPLE.  > 


Routine  4:  Forced  choice  of  axampla 


r 


One  of  tha  instencat  below  it  an  example  of  e sycophant. 

The  other  is  not. 

1 ) a handsome  young  man  complamenting  en 
older  women  because  he  respects  end  edmiras 
her 

\ 

2)  e handsome  young  men  flattaring  a rich  older 
woman  in  order  to  be  written  into  her  will 

Type  the  number  of  the  inttence  that  it  an  axampla.  > 


Routine  6.  Practice 


r \ 

If  this  instance  is  en  exampla  of  e sycophant,  typa  the 
word:  SYCOPHANT.  If  it  is  not  an  axampla  of  a 
sycophant,  type:  NOT  A SYCOPHANT. 

A second-string  football  player  epple-polishing  the 
coaches  in  order  to  get  e starting  position  on  the  teem 


Figure  1.  Simulated  screen  display  for  DEFINE  and  CLASSIFY, 
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choice  paraphrasing  occurs  in  Routine  5.  This  and  other  options 
in  other  modules  alias?  for  variability  in  what  we  believe  to  be 
important  instructional  parameters.  These  options  alio?  us  to 
make  some  modules  more  similar  or  less  similar  to  each  other  and 
to  isolate  parameters  that  theoretically  might  produce  signifi- 
cant differences  in  achievement. 

Routine  6;  Multiple  Choice  Definition  Construction.  In 
this  routine  the  general  statement  of  the  form  of  the  definition 
(from  Routine  4)  is  given.  The  student  selects  a correct  para- 
phrase (different  from  those  in  Routine  5)  for  all  of  the  parts 
of  the  definition.  This  routine  thus  begins  to  approximate 
definition  construction.  Feedback  for  an  incorrect  response 
directs  the  student  to  try  again.  After  two  incorrect  re- 
sponses, the  correct  definition  is  shewn. 

Routine  7:  Definition  Construction.  In  the  final  routine 
of  this  module  the  student  is  instructed  to  type  a definition 
of  the  target  word.  She/he  is  cued  to  start  by  typing  the 
target  word  and  to  phrase  the  definition  appropriately  for  a word 
in  its  form  class.  After  making  the  response,  the  student  com- 
pares his  or  her  definition  with  a presentation  of  the  original 
definition. 

The  criterion  test  for  this  module  is  similar  to  Routine  7 , 
definition  construction,  minus  the  feedback. 

For  the  remaining  modules  only  those  routines  that  are  dif- 
ferent from  the  routines  in  the  DEFINE  module  will  be  described. 
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Classify  Module 


Routine  3:  Exanple/Non-exanple . After  presenting  the  word 
and  its  pronunciation  plus  the  definition,  CLASSIFY  next  pre- 
sents a situational  exanple  of  the  target  word  and  a matching  nan- 
example  that  differs  from  the  exanple  on  only  one  attribute. 
Exanple  and  non-example  are  labeled  and  a question  is  asked  which 
requires  the  student  to  attend  to  the  presence  of  the  attribute 
in  the  exanple  and  its  absence  in  the  non-example.  The  student 
responds  by  typing  either  EXAMPLE  or  NCMEXAMPIE.  Feedback  for 
an  incorrect  response  calls  attention  to  the  attribute  the  stu- 
dent should  have  attended  to  in  making  the  response,  then  directs 
the  student  to  choose  again.  This  routine  repeats  for  each 
attribute  of  the  target  word  then  a final  pair  is  presented  which 
varies  all  attributes  at  once.  (Figure  1 presents  screen  dis- 
plays for  CLASSIFY  Routines  3,  4 and  C) . 

Routine  4;  Forced  Choice  of  Exanple.  Next  a matched 
exanple/non-exanple  pair  of  situations  is  presented  unlabeled. 

The  student  is  instructed  that  only  one  of  the  instances  is  an 
example  of  the  target  vrord  and  is  asked  to  identify  it  by  typing 
its  number.  Feedback  indicates  whether  the  response  is  correct 
or  not  and  if  not,  the  specific  wording  in  the  exanple  that 
"gives  it  away"  as  the  example  is  capitalized.  The  student 
chooses  again.  Note  that  in  both  Routines  3 and  4 the  student 
must  make  the  correct  response  before  receiving  the  next  item. 
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This  routine  is  repeated  until  a total  of  six  or  seven  exanple/ 
non-exanple  pairs  have  been  presented  in  both  Routines  3 and  4. 

Routine  5:  Forced  Choice  Exanple  Stannary.  This  routine 
re-presents  only  the  examples  frcrn  the  previous  routine.  They 
are  identified  as  examples  and  the  student  is  instructed  to  note 
how  they  are  alike  and  how  they  differ. 

Option:  The  short  form  of  the  definition  can  remain  on  the 
screen  during  Routines  3,  4 and  5.  Having  the  definition  on  makes 
this  module  more  similar  to  DEFINE. 

Routine  6 : Practice . In  the  final  routine  the  student  is 
presented  with  one  instance  at  a time  with  the  direction  to  type 
the  target  word  if  the  instance  is  an  exanple  or  type  "Not  (target 
word)"  if  it  is  a non-example.  This  routine  is  repeated  four  times 
with  the  instances  being  randomly  varied  between  examples  and  nan- 
examples  with  the  constraint  that  at  least  one  exanple  and  non- 
exanple  shall  appear  among  the  four  instances.  Feedback  for  an 
incorrect  response  labels  the  given  instance  as  either  EXAMPLE 
or  NCNEXAMPLE  and  presents  the  matching  instance.  The  student 
is  directed  to  read  and  compare  the  two  instances.  This  routine 
re-presents  missed  items  after  all  items  have  been  presented  once. 

The  criterion  test  for  this  module  is  similar  to  Routine  6, 
minus  the  feedback  and  the  re-presenting  feature. 

Discover  Module 

This  is  a discovery  version  of  CLASSIFY,  the  rationale  being 
that  a task  requiring  the  student  to  induce  a concept  from 
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exarrples  might  aid  in  the  development  of  the  critical  attributes 
in  memory.  The  routines  are  largely  the  same  as  in  CLASSIFY  but 
there  are  two  important  differences.  First,  the  definition  is  not 
presented  until  the  end  in  DISCOVER.  Second,  in  place  of 
CLASSIFY' S inquisitory  exanple/non-exanple  routine,  DISCOVER  has 
an  expository  Introductory  Examples  routine  which  simply  presents 
a labeled,  matched  pair  without  calling  attention  to  the  critical 
attributes  in  the  example. 

The  DISCOVER  module  then  goes  into  the  Forced  Choice  of 
Exairple  routine,  identical  to  that  in  CLASSIFY,  and  then  into  the 
Practice  routine. 

Options:  The  example/non-exanple  pair  fran  the  Introductory 
Example  routine  and  those  pairs  frcm  the  Forced  Choice  of  Examples 
routine  can  remain  on  the  screen  throughout  the  latter  routine. 

If  this  option  is  chosen,  then  the  Practice  routine  is  identical 
to  Practice  in  CLASSIFY  (the  examples  having  been  erased  from  the 
screen) . If  the  option  is  not  chosen  then  Practice  takes  a slightly 
different  form.  Instead  of  presenting  only  four  instances  to  be 
classified  and  reserving  their  matching  instances  for  feedback, 
the  matching  instances  are  included  in  the  instance  pool,  thus 
giving  four  example/non-exanple  pairs,  randomly  arranged.  Feed- 
back for  an  incorrect  answer  simply  indicates  that  the  answer  is 
wrong  and  directs  the  student  to  try  again. 
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The  final  routine  presents  the  short  version  of  the  definition 
with  attributes  labeled. 

The  criterion  test  is  the  same  as  for  CLASSIFY. 

Word  Relations  Module 

This  module  begins  with  the  pronunciation  and  definition 
routines  described  earlier.  The  third  routine,  Forced  paraphrase, 
is  similar  to  the  Multiple  Choice  Paraphrase  routine  in  DEFINE 
except  that  instead  of  paraphrasing  each  separate  attribute  a 
paraphrase  of  the  target  word  is  selected.  (Figure  2 presents 
screen  displays  for  WORD  RELATIONS  Routines  3,  5 and  6) . 

Insert  Figure  2 about  here 

The  characteristic  routines  for  this  nodule  begin  with 
Routine  5,  Explanation  of  Relations.  This  routine  illustrates 
how  two  words  can  be  related  to  each  other  by  presenting  two 
examples  of  a particular  relation.  The  first  exaitple  is  an  arche- 
type of  the  relation,  e.g. , "Helpfulness  is  a QUALITY  of  a Boy 
Scout."  The  second  example  illustrates  the  same  relation  for 
the  target  word,  e.g.,  "Insincerity  is  a QUALITY  of  a sycophant." 
The  student  is  then  directed  to  select  another  word  that  bears 
the  same  relation  to  the  target  vrord.  This  routine  is  repeated 
for  three  or  four  different  relations.  Feedback  identifies  the 
response  as  correct  or  incorrect  and  if  incorrect,  the  student 
is  directed  to  try  again. 
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WORD  RELATIONS 


WORDS  IN  CONTEXT 


Routine  3:  Paraphrase 


Ona  of  these  phrases  captures  tha  meaning  of 
SYCOPHANT. 


1 )  a hard-working  2)  a fast-talking  3)  a wealthy 
achiever  apple-polisher  fat  parson 

Type  the  number J> 


V  J 

Routine  5:  Relations 

( 'N 

Words  can  be  ralavad  to  each  other  in  various  ways. 

Por  example, 

if  helpfulness  - is  a QUALITY  - of  a Boy  Scout, 
then  insincerity  - is  a QUALITY  - of  a sycophant. 

Fill  in  tha  blank. 

I**??**)  could  also  be  a QUALITY  of  a sycophant. 

1)  honesty  21  kindness  3)  ambition  4)  fairness 

> 

V  J 

Routine  6:  Practice 

( \ 

Identify  tha  relationship  by  typing  tha 
correct  number. 

How  Is  charm  related  to  a sycophant? 

1)  is  a JOB  of  2)  is  a TOOL  of  3)  is  a GOAL  of 

> 


Routine  4:  Story  and  Questions 

f 

King  Foolish  liked  to  be  surrounded  by  members 
of  his  court.  Ha  spent  his  whole  day  listening  to 
them  flatter  him  to  build  up  isis  ago. 

One  of  tha  attendants,  who  thought  ha  might  be 
rewarded  by  tha  king  with  a purse  full  of  gc'd,  wss 
full  of  praise.  This  attendant  was  always  saying, 

"King  Foolish,  how  wonderful  you  eral  King  Foolish, 
you're  tha  wisest  ruler  a kingdom  could  hevel  King 
Foolish,  you're  tha  greatest!" 

Another  attendant  wanted  to  be  made  a knight. 

This  attendant  had  a slightly  different  approach  to 
winning  tha  king's  favor.  He  was  constantly  kneeling 
before  tha  king  end  kissing  his  hand.  Whan  tha  king 
was  around,  this  attendant  never  got  off  his  knees. 

Tha  queen  watched  whet  was  going  on  among  tha 
membert  of  the  court.  "Humphl"  she  snorted  ona 
day  whan  she  was  fed  up.  "They're  sycophants,  all  of 
them.  Each  of  those  attendants  is  about  as  sincere  as 
my  pet  monkey." 

Tha  quean  called  tha  attendants  sycophants  because: 

1)  they  buttered  up  the  king  to  get  fevors. 

2)  they  flattered  tha  king  to  make  her  jealous. 

3)  they  praised  the  king  to  meke  him  heppy. 

> 

V ) 

Routine  8:  Sentence  Completion 


A student  might  be  called  a sycophant  if  he 


Press  return  whan  you  have  finished. 


Compare  your  answer  with  this  sample  answer. 

A student  might  be  called  a sycophant  if  he 
flattered  the  teacher  in  order  to  get  a high 
grade. 


Complete  this  sentence  in  your  own  words. 


Figure  2.  Simulated  screen  display  for  WORD  RELATIONS  and  WORDS  IN  CONTEXT. 


Opt’  ~>n:  The  sho’i;  form  of  the  definition  can  remain  on  the 
screen  during  Routines  3 and  5. 

In  Routine  6 a word  is  presented  aJong  with  the  target  word 
and  the  student  selects  the  name  of  the  relation  between  the  two 
words.  Hois  routine  continues  through  several  examples  for  each 
of  several  relations,  different  ones  from  those  used  in  Routine 
5.  In  writing  instructional  materials  for  this  module  we  work 
from  a list  of  15  possible  relations . The  list  includes,  for 
example:  CAUSE  of,  RESULT  of,  TOOL  of,  TARGET  of,  IOCMTCN  of, 
PART  of,  etc.  Not  all  the  relations  are  applicable  to  each 
target  word,  of  course,  but  for  those  that  apply  four  or  five  ex- 
anples of  each  relation  are  written.  This  routine  repeats  for 
about  20  questions,  presenting  2 to  4 questions  for  each  appro- 
priate relation.  Feedback  for  an  incorrect  response  identifies 
the  answer  as  wrong  and  directs  the  student  to  try  again.  Missed 
items  are  re-presented  at  the  end  of  the  series. 

The  final  routine,  Relation  Summary,  lists  all  the  relations 
taught  for  the  particular  target  word  and  re-presents  an  example 
of  each  relation.  The  student  is  required  to  type  the  target 
word  several  times  in  this  routine. 

The  criterion  test  is  similar  to  Routine  6,  minus  feedback. 
Because  this  test  is  based  upon  a model  of  memory,  we  consider 
it  to  be  experimental  in  nature.  It  will  be  of  great  interest 
to  us  to  determine  whether  a subject  receiving  instruction  solely 


via  WORD  RELATIONS  will  reflect  a difference  in  cognitive  structure 
fran  these  subjects  receiving  instruction  via  the  other  modules. 

Word  in  Context 

The  characteristic  routines  in  this  module  present  a story 
of  about  160  words  and  a series  of  six  or  zjven  questions  which 
relate  details  of  the  story  to  attributes  of  the  target  word 
(see  Figure  2) . The  target  word  appears  in  the  story  only  once, 
toward  the  end.  The  stories  are  often  generated  fran  one  of  the 
examples  fran  CLASSIFY.  As  an  aid  in  generating  the  questions 
Stein  and  Glenn's  (1977)  schema  for  analyzing  story  catprehension 
(derived  fran  Rumelhart)  served  as  a guide  in  selecting  those 
units  of  a story  that  are  most  likely  to  be  remembered. 

The  questions  vary  in  format:  multiple  choice,  yes/no  and 
short  answer.  Feedback  for  an  incorrect  response  identifies  the 
answer  as  wrong  and  directs  the  student  to  try  again. 

Option:  The  story  can  remain  on  the  screen  while  the  ques- 
tions are  presented.  If  this  option  is  chosen,  feedback  for  an 
incorrect  response  also  includes  underlining  the  pertinent  words 
of  the  story. 

The  last  question  presented  is  in  the  form  of  a multiple 
choice  sentence  completion.  This  approximates  the  terminal  task 
of  the  module. 

The  final  routine  requires  the  student  to  finish  a sen- 
tence, the  giver,  part  of  which  includes  the  target  word.  The 


student  response  calls  for  an  explanation  of  the  word.  For  example, 
"A  student  might  be  called  a sycophant  if  he  . . 

The  criterion  test  for  this  module  is  sentence  completion. 

Word  Line  Module 

This  module  is  identical  to  WORD  RELATIONS  for  the  first 
four  routines.  The  first  new  routine  for  this  module,  Routine 
5 Illustrative  Line,  introduces  the  student  to  a word  line, 
demonstrates  how  it  is  used  and  gives  the  student  brief  practice 
on  the  task  that  will  follow.  After  having  done  this  routine 
once,  the  student  has  the  option  of  skipping  it  the  next  time 
he/she  works  through  this  module.  {Figure  3 presents  screen 
displays  for  WORD  LINE  Routines  5,  6,  7 and  8) . 


Insert  Figure  3 about  here 


In  Routine  6,  Map  Line,  the  word  line  presents  the  target 
word  in  a group  with  five  other  words  of  the  same  form  class  and 
requires  the  student  to  arrange  them  along  the  line  according  to 
some  semantic  dimension  (usually  the  major  attribute  of  the  target 
word  drawn  from  the  definition) . In  Figure  3,  for  example,  the 
dimension  of  the  word  line  is:  to  make  larger/to  make  snaller, 
for  the  target  word  "augment."  The  other  words  in  the  group  in- 
clude one  synonym  of  the  target  word,  two  antonyms  and  two  words 
that  fall  into  the  middle  ground  along  the  given  dimension.  Note 
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Routine  5:  Illustrate  Wordllne 


Routine  7:  Reteline 


Routine  6:  Mepllne 


Routine  8:  Line  Summery 


Figure  3,  Simulated  screen  display  for  WORDUNE. 


that  the  instructions  avoid  implying  that  the  word  line  repre- 
sents a continuum  with  subtle  gradations.  The  task  essentially 
requires  the  students  to  sort  six  words  into  three  categories ; 
in  this  case,  the  categories  are  to  make  smaller,  to  make  larger 
and  to  do  something  that  neither  makes  smaller  nor  larger.  TVo 
words  go  into  each  category  and  the  position  of  the  words  within 
categories  is  not  important.  Feedback  identifies  the  response 
as  correct  or  incorrect  and  places  the  word  on  the  appropriate 
section  of  the  line. 

In  Routine  7,  Rate  Line,  the  student  is  given  a short  word 
line  with  the  target  word  and  one  other  word  in  each  category 
in  position  on  the  line.  Six  instances  are  presented  and  the 
student  must  select  their  appropriate  positions  on  the  line,  thus 
relating  an  instance  to  a word  on  the  line. 

In  Routine  8,  the  final  routine,  questions  are  asked  and 
the  student  responds  with  the  words  frcm  the  word  line.  This 
provides  two  opportunities  for  the  student  to  write  the  target 
word  (plus  the  earlier  opportunity  in  the  Type  frcm  Memory 
routine) . The  criterion  test  for  WORD  LINE  is  similar  to  Routine 
6,  the  student  arranges  the  target  word  and  five  other  words  on 
a line. 

Equivalents  Module 

The  Equivalents  module  is  a very  simple  comparison  module. 

It  consists  of  a single  routine  which  presents  repeated  practice 


on  items  missed  cn  the  multiple  choice  pretest.  The  format  is 
the  same  as  on  the  pretest  except  for  feedback.  Confirmation 
is  given  following  a correct  response;  in  case  of  a wrong  response, 
the  correct  answer  is  identified.  Items  continue  to  be  re- 
presented until  the  student  has  responded  correctly  a given  number 
of  consecutive  times.  The  option  for  this  module  is  that  the  num- 
ber of  consecutive  correct  answers  required  may  vary.  An  important 
feature  of  this  module  is  the  use  of  an  item  presentation  strategy 
that  attempts  to  maintain  a lag  of  three  interpolated  items  between 
a word's  and  (n  + 1)^*  presentation. 

Insert  Figure  4 about  here 

Figure  4 illustrates  the  computer  management  of  the  instruc- 
tional system.  The  left  column  (Input)  refers  to  the  various 
computer  programs  that  process  the  instructional  material  for 
presentation  to  the  student  on  the  DECscope.  The  output  pro- 
gram collects  and  prints  out  the  student  response  data.  The 
Student  Use  Package  refers  to  the  programs  that  test  the  student 
and  present  the  words  missed  on  the  pretest  for  instruction  via 
one  or  more  modules. 

There  will  eventually  be  two  modes  of  system  use;  Instruc- 
tional and  Experimental.  At  present  we  are  in  the  Experimental 
mode.  In  this  mode  the  experimenter  specifies  how  many  words 
the  student  will  learn  and  which  module  or  modules  will  be 
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Figure  4.  CAI  Vocabulary  Instructional  System 


needed  for  instruction.  Later,  in  the  instructional  mode, 
prescriptions  specifying  combinations  of  modules  will  be  developed 
from  the  experimental  findings  and  tested  for  optimum  achievement. 

The  Oorputer  System 

Hie  vocabulary  instruction  is  coded  in  Fortran  IV  and  will 
be  delivered  via  Digital  Model  VT-52  Video  Display  terminals  with 
a 120  character/sec.  transmission  rate,  connected  to  a PIP-15. 

The  time-sharing  system,  ETSS,  was  developed  at  the  Learning  Re- 
search and  Development  Center  (Fitzhugh,  1973) . The  system  con- 
sists of  a DEC  PDP-15  computer,  over  100  megabytes  of  high  speed 
disk  memory,  and  a terminal  controller  capable  of  supporting 
up  to  32  work  sites.  In  addition,  a small  DEC  PDP-11  canputer  is 
attached  to  the  PDP-15  and  controls  a high  speed  line  printer, 
a card  reader  and  a card  punch.  An  on-line  text  editor,  a com- 
prehensive statistical  package  and  a variety  of  utility  systems 
are  also  provided.  Several  applications  programers  develop  the 
instructional  programs  according  to  the  specifications  of  the 
instructional  designers. 

Since  our  CAI  programs  are  not  yet  completed , the  studies 
we  report  were  conducted  with  paper  and  pencil  materials.  In 
the  second  study,  we  attenpted  to  achieve  as  much  moment-to-manent 
control  over  student  learning  processes  as  was  possible  within  the 
constraints  of  a relatively  inflexible  paper  and  pencil  media. 
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Two  Vocabulary  Studies 


Two  studies  were  conducted  in  pursuit  of  determining  the 
instructional  effects  of  several  modules. 

Study  I 

The  first  study  (Rosner,  1978)  used  materials  and  a pro- 
cedure directly  modeled  upon  the  1966  study  noted  previously 
that  served  as  our  initial  empirical  justification  for  the  exis- 
tence of  indirect  effects  from  vocabulary  insturctional  strate- 
gies. The  purpose  of  Rosner' s study  was  in  part  to  determine 
whether  the  sizeable  indirect  effects  found  with  the  use  of  a 
definitions  method  could  be  replicated  in  our  laboratory.  She 
taught  four  difficult  vocabulary  words  that  were  identified  on 
the  basis  of  a pilot  study  and  word  frequency  counts  to  college 
students.  Unfortunately,  despite  the  fact  that  the  difference 
between  the  number  of  words  her  instructional  groups  mastered 
compared  to  her  control  group  was  comparable  to  the  difference 
between  instruction  and  control  groups  in  the  1966  study,  her 
difference  was  not  significant.  Thus,  it  was  not  meaningful  to 
discuss  the  effects  obtained  via  the  instruction. 

We  suspect  that  most  of  the  difficulties  of  the  study  were 
procedural.  First,  students  in  the  1966  study  kept  the  materials 
for  12  minutes;  Rosner 's  students  paced  themselves  for  an  average 
of  7 minutes  with  the  material.  Recent  research  on  learning 
within  CM  or  OH  environments  has  shown  that  problems  students 


enoounter  center  around  deficiencies  in  sel f -management  skills 
(see  Judd,  McCombs  & Dobrovolny,  1978)  and  they  need  skill 
management  training.  It  seems  likely,  therefore,  that  students 
simply  did  not  manage  their  word  study  effectively.  A second 
problem  is  concerned  with  defining  an  appropriate  degree  of  con- 
trol over  student  learning  processes  when  only  a single  frame  is 
devoted  to  a particular  word  (as  was  the  case  in  the  1966  study 
for  the  definitions  method) . This  level  of  instructional  control 
over  student  learning  processes  is  fairly  gross;  students  can 
make  the  single  required  response  for  each  instructional  word, 
then  attention  can  wander  for  the  rest  of  the  interval  (in  same 
students)  or  appropriate  study  behavior  will  take  place  (in 
others) . Perhaps  it  is  the  case  that  the  naturally  occurring 
distribution  of  learner  processes  during  a "free"  period  is  the 
significant  determiner  of  achievement  and  not  instructional 
method  (at  least  as  such  methods  were  implemented  in  the  1966 
study).  Since  Posner's  learners  spent  a briefer  time  in  instruc- 
tion than  those  in  the  1966  study,  it  is  possible  that  had  more 
time  been  required  (or  multiple  frames  per  word  been  used)  achieve- 
ment might  have  been  increased.  The  lack  of  significant  differ- 
ences may  have  been  due  to  a higher  level  of  prior  learning  of 
the  instructional  words  than  was  expected.  Finally,  it  is  ccmmonly 
found  that  undergraduate  students  are  not  often  motivated  to  per- 
form well  in  psychological  experiments.  We  expect  that  the 
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responsive  instruction  provided  by  our  CAI  will  induce  more 
substantial  levels  of  learning. 

We  improved  the  methodology  of  the  second  study  by  using 
multiple  frames  to  instruct  a word  and  the  frames  were  modeled 
after  tvro  of  our  CAI  modules.  This  change  had  the  effect  of 
increasing  instructional  time  and  theoretically,  achieved  more  pre- 
cise control  over  student  learning  processes  within  a given  in- 
structional interval.  We  also  used  a pretest  as  a method  for 
identifying  previously  unknown  instructional  words.  Thus,  our 
second  study  is  modeled  as  closely  after  our  instructional  system 
design  as  we  could  accomplish  within  the  constraints  of  paper 
and  pencil  materials . A second  constraint  was  that  we  had  to  use 
a fixed  number  of  instructional  sessions  because  our  subjects 
were  recruited  from  a small  educational  psychology  class. 

Study  II 

Purpose.  The  major  purpose  of  the  study  was  s imply  to 
verify  the  existence  of  indirect  effects  under  the  instructional 
conditions  we  had  designed.  The  modules  tested  were  those  for 
which  material  had  been  written  and  bested,  the  DEFINE  module 
and  the  CIASSIFY  module  (in  adapted  paper  and  pencil  form) . In 
the  paper  and  pencil  adaptation,  we  used  all  of  the  instructional 
routines  that  defined  each  module,  adapted  as  best  we  could  to 
a paper  and  pencil  programed  instructional  format.  The 
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instructional  materials  consisted  of  a series  of  frames  for  each 


word  that  followed  a shaping  procedure. 

Design  and  Procedure.  Ten  infrequent  words  were  taught  to 
two  groups  of  students  recruited  from  an  introductory  educational 
psychology  class.  There  were  12  students  randomly  assigned  to 
the  two  instructional  groups.  The  materials  for  each  instructional 
word  consisted  of  five  pages  of  instructional  frames  with  response 
feedback  typically  at  the  top  of  a "next”  page.  Students  also 
had  to  use  a paper  cover  (laminated  plastic  sheet)  designed  to  be 
used  to  cover  up  certain  portions  of  the  previous  text  material. 

The  paper  cover  was  our  attenpt  to  mimic  the  erasure  capability 
of  a CRT.  The  words  were  presented  in  a different  random  order 
to  the  students. 

In  the  first  session,  students  took  a multiple  choice  pre- 
test requiring  identification  of  a verbal  phrase  equivalent  in 
meaning,  learned  the  ten  words  (some  of  which  they  already  knew 
as  indicated  by  the  pretest)  and  then  took  an  immediate  definitions 
post- test.  It  was  not  possible  given  constraints  under  which  the 
study  operated  to  teach  students  cnly  the  words  they  missed. 
Students  returned  exactly  one  week  later  for  retention  tests 
(they  were  not  informed  they  would  be  tested) . They  were  given 
six  tests  - free  recall,  definition-stating,  sentence  generation, 
sentence  understanding,  classification;  the  multiple  choice  pre- 
test also  served  as  a post- test.  The  free  recall  test  was 
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included  because  we  believe  nodules  other  than  the  ones  used  in 
the  present  study  will  influence  free  recall,  and  we  wanted  a 
measure  fran  the  two  methods  of  interest  in  this  study.  The  tests 
were  ordered  as  noted  to  minimize  learning  fran  one  test  to  the 
next. 


Results  and  Discussion 

Time  and  totcil  learning.  There  was  a significant  difference 
in  the  mean  amount  of  time  spent  learning  in  the  two  groups. 

Mean  time  spent  in  Define  instruction  was  slightly  longer  (X  = 

29.4  minutes)  than  in  Classify  instruction  (X  = 25.7  minutes) , 
a difference  of  3.7  minutes  (t  = 2.40,  df  = 22,  p < .05).  We  do 
not,  however,  think  the  time  difference  is  large  enough  to  be 
practically  significant  or  to  strongly  bias  our  achievement 
results.  Our  mean  time  per  word  was  close  to  the  1966  study 
which  used  12  minutes  for  four  words. 

The  two  groups  did  not  differ  in  prior  level  of  word 
knowledge  when  entering  the  instructional  treatments;  the  Define 
group  knew  an  average  of  6.7  vrords,  while  the  Classify  group  knew 
an  average  of  5. 8 words  (t  = 1.25,  df  = 22,  p > .05) . A t test 
of  the  mean  number  of  words  gained  in  each  group  using  scores  an 
the  multiple  choice  test  (a  test  of  general,  vocabulary  knowledge) 
that  was  given  as  a pre  and  post-test  shewed  both  groups  gained 
an  equivalent  amount  (X  gain  for  Define  was  2.67;  X gain  for 
Classify  was  3.10;  t = 2.074,  df  = 22,  p > .05) . The  Define  group 
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obtained  94%  (6.7  + 2.7)  overall,  while  the  Classify  group  ob- 
tained 79%  (5.8  + 2.1) . At  test  of  the  scores  earned  on  the 
multiple  choice  post-test  showed  the  means  were  not  signifi- 
cantly different  (t  = .921,  df  = 22,  p > .05). 

j 

A comparison  of  s acres  on  the  inmediate  definitions  test 
with  the  delayed  definitions  test  (see  sooring  system,  p.  30) 

i 

revealed  higher  average  immediate  scores  for  both  groups  and  no 
significant  difference  between  the  two  groups  in  proportion  of 
new  words  learned,  then  forgotten  (t  = .549,  df  = 22,  p > .05) . 
There  were  no  differences  in  the  number  of  instructional  words 
that  were  recalled  in  the  free  recall  test  (X  for  Define  was 

I 

4.0;  X for  Classify  was  3.9;  t * .11,  df  * 22) . These  data  are 
decidedly  consistent  with  the  proposition  that  in  terms  of  total 
amount  of  achievement,  type  of  instructional  method  does  not 
make  a difference. 

I 

i 

Indirect  Effects 

The  concepts  of  direct  and  indirect  effects  require  that 
we  consider  instructional  methods  not  only  in  terms  of  contri- 
bution to  an  outcome  of  general  interest  but  in  terms  of  patterns 
of  achievement  across  multiple  specific  objectives  that  might  be 
desired  as  outcomes  of  an  instructional  program. 


Insert  Tables  2 and  3 about  here 


Mean  Performance  on  Several  Criterion  Tests  of  the  Instructional  Objectives. 
The  Proportions  in  Parenthesis  are  the  Comparable  Figures  from  the  1966  Study 


Tables  2 and  3 shew  the  mean  performance  on  several  criterion 
tests  of  the  instructional  objectives  as  a function  of  instructional 
method  (the  sentence  understanding  test  is  not  included  as  we  have 
not  arrived  at  a reliable  scoring  system  yet) . The  data  are  pre- 
sented for  all  instructional  words  (Table  2)  and  for  only  those 
words  learned  in  the  course  of  the  instruction  (the  "new"  words 
[Table  3])  . 

The  total  number  of  points  that  could  be  earned  varied  from 
one  criterion  test  to  another.  The  definitions  post- tests  was 
scored  0,  1,  2 depending  on  number  of  attributes  mentioned  in  a 
definition  (reliability  = .90;  maximum  points  = 12x10x2  = 240); 
the  classification  test  comprised  8 instances  per  word  (maximum 
points  = 12x10x8  = 960) ; the  sentence  generation  test  was  scored 
0,  1,  2,  3 depending  on  meaning  and  usage  (reliability  = .90; 
maximum  points  = 12x10x3  = 360) . The  data  for  all  words  in 
the  Define  group  are  clearly  comparable  to  the  1966  study?  there 
are,  however,  some  discrepancies  in  the  Classify  group.  The 
discrepancy  in  the  sentencing  test  is  easily  explained;  our  test 
required  complete  sentence  construction  (using  the  word  "because") 
whereas  the  1966  study  required  sentence  c Tpletion  (a  meaningful 
stem  was  given) . There  were  procedural  differences  in  concept 
teaching  methods  between  the  1966  study  and  ours  which  is  probably 
responsible  for  our  classify  students  lewer  defining  perfon  ice. 

The  data  for  new  words  only  shew  lowered  performance  on  the  defining 
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and  sentencing  tests  as  would  be  expected  for  words  that  students 
have  no  prior  knowledge  of.  Hie  pattern  of  levels  of  achievement 
shown  for  all  words  is  reflected  in  the  new  words  only  for  the 
defining  end  sentence  generating  tests.  The  classification  test 
retains  a high  level  of  performance  because  it  is  an  easier  test. 

A test  of  the  largest  achievement  difference  (Define  vs.  Classify 
for  all  words)  revealed  no  significant  differences:  t = 2.006, 
df  - 22,  p > .05.  We  are  tempted  to  conclude,  as  was  done  in  the 
1966  study,  that  transfer  fran  one  instructional  method  to  tests 
based  on  other  methods  is  100%  and  therefore  we  have  evidence  of 
substantial  indirect  effects. 

Within  a CAI  context,  however,  it  is  standard  procedure  to 
define  criteria  that  individuals  must  meet  before  they  can  exit 
a particular  segment  of  instruction.  Both  studies,  ours  and  the 
1966  study,  fixed  the  amount  of  time  and  material  to  be  covered 
within  an  instructional  treatment,  and  did  not  set  a mastery 
criterion.  Therefore,  in  order  to  studv  the  indirect  effects 
that  may  be  associated  with  a particular  instructional  strategy, 
we  must  assess  those  effects  contingent  upon  a performance  standard. 

Insert  Tables  4 and  5 about  here 

Tables  4 and  5 show  the  quality  of  performance  on  the  several 
criterion  tests  as  a function  of  the  quality  of  performance  on  the 
criterion  test  matched  to  the  instructional  method.  In  each 
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TABLE  4 


Quality  of  Performance  on  the  Classification  and  Sentence 
Generation  Posttests  as  a Function  of  Quality  of 
Performance  on  the  Definitions  Posttest  for 
the  DEFINE  Instructional  Group. 


Type  of  Performance 
on  Define  Post 


Number  of 

Subj  ects 

Frequency 

Good 

8 

18 

Poor 

5 

14 

Type  of  Performance 


on  Other  Posts 

Sentence 

Classify Generation 


Good 

P 

Good 

P 

18 

1.0 

16 

.89 

8 

.57  • 

3 

.21 

TABLE  5 


Quality  of  Performance  on  the  Definitions  and  Sentence 
Generation  Po'sttests  as  a Function  of  Quality  of 
Performance  on  the  Classify  Posttest  for  the 
CLASSIFY  Instructional  Group. 


Type  of  Performance  Type  of  Performance 
on  Classify  Post  on  Other  Posts 


Define 

Sentence 

Generation 

Number  of 

Frequency 

Good 

? 

Good 

P 

Subjects 

Good 

12 

38 

10 

.27 

16 

.42 

Poor 

7 

12 

2 

.16 

2 

.16 

instructional  group  for  each  new  word  and  each  student,  the 
occasions  cn  which  the  post-test  score  was  excellent  (a  score  of 
2 for  the  definitions  pose;  a score  of  7 or  8 instances  classi- 
fied correctly  out  of  8 instances)  and  the  occasions  on  which 
it  was  poor  (a  score  of  0 for  the  definitions  post;  a score  of 
less  than  7 instances  classified  correctly)  were  noted.  For 
these  two  classes,  observations  of  the  quality  of  performance  of 
the  same  student  cn  the  same  wurd  were  made  on  the  other  post- 
tests. For  exanple,  in  Table  4,  of  the  18  observations  of  high 
quality  definitions,  there  were  18  observations  of  high  quality 
classification  performance  and  16  observations  of  high  quality 
(a  score  of  3)  sentence  generation.  It  can  be  easily  seen  from 
a comparison  of  Tables  4 and  5 that  the  consistency  in  the 
quality  of  performance  given  cn  a defining  post  (Table  4)  is  quite 
different  than  that  given  an  a classifying  post  (Table  5) . The 
trends  evident  in  the  tables  tend  to  show  that  Define  instruc- 
tion leads  to  rrore  direct  effects  than  Classify  instruction, 
for  high  quality  performances  tend  to  be  maintained  to  a greater 
degree.  However,  a test  of  the  significance  of  the  difference 
in  the  proportions  using  the  normal  approximation  to  the  Irwin- 
Fisher  exact  test  shewed  no  significant  differences  (x  =2.39, 
df  = 3,  p > .05).  While  the  construction  of  prescriptions  is 
more  interesting  when  methods  produce  differently  sized  indirect 
effects,  our  preliminary  data  have  suggested  the  indirect  effects 
may  be  similar  rather  than  different  in  magnitude. 
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We  have,  therefore,  decided  that  within  a CAI  context  it  is 
more  appropriate  to  assess  indirect  effects  under  conditions  of 
instructional  treatments  with  pre-defined  .proficiency  levels 
rather  than  ccnditicns  defined  in  terms  of  a fixed  amount  of 
time  or  number  of  items.  This  does  not  negate  the  formulation 
presented  previously.  What  we  have  done  is  to  clarify  the  defi- 
nition of  indirect  effects  in  a criterion  referenced  (rather 
than  norm  referenced)  context.  The  design  of  our  CAI  system 
reflects  this  clarification:  We  have  incorporated  an  interim 
test  for  each  module  whose  format  is  matched  to  the  module.  Stu- 
dents cannot  exit  from  a given  module  until  the  interim  test  is 
passed. 

Future  Work 

In  the  near  future,  our  research  will  concentrate  on  testing 
for  indirect  effects  and  testing  predictions  frcm  our  model,  such 
as  theorem  number  7 mentioned  previously.  Our  goal  is  to  develop 
the  prescriptive  rules  that  allcw  us,  through  suitable  selection 
of  instructional  strategies,  to  optimize  a given  performance 
function  across  a variety  of  criterion  tests,  within  certain 
time  constraints.  As  we  perform  our  research  we  will,  of  course, 
be  providing  services  to  developmental  reading  laboratories  and 
we  are  anxious  to  begin  implementation  at  our  planned  sites. 
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